Video deblocking

ABSTRACT

Video deblocking can be implemented in video content processing and delivery environments when displaying decompressed/decoded video information. Discontinuities are identified and smoothed to reduce blockiness, for example at macroblock boundaries in conjunction with Moving Picture Expert Group (MPEG)-based schemes. In exemplary implementation(s), a client device detects discontinuities at or near block boundaries, determines which discontinuities are artificial, and smooths at least those discontinuities that are determined to be artificial. These actions may be accomplished using (i) one or more Laplacian of Gaussian (LoG) operations as applied to both an image and a quantization matrix and (ii) windowed comparisons between values and a threshold resulting from the LoG operations. Alternatively, these actions may be accomplished using a spatio-temporally varying filter. Additionally, in an MPEG context, intra image units may be deblocked using the former approach while non-intra image units may be deblocked using the latter approach.

RELATED PATENT APPLICATION

This U.S. Nonprovisional Patent Application is a Divisional Applicationof co-pending U.S. Nonprovisional patent application Ser. No.10/179,825, filed on Jun. 24, 2002, and entitled “Video Deblocking”.Application Ser. No. 10/179,825 is hereby incorporated by reference inits entirety herein.

TECHNICAL FIELD

This disclosure relates in general to video deblocking and inparticular, by way of example but not limitation, to post-processing ofvideo information to reduce blockiness in the display thereof.

BACKGROUND

Television-based entertainment systems are expanding the programming andservices that they offer. In addition to television program content suchas that found on broadcast and traditional cable networks, televisionservice providers are adding on-demand video, as well as otherinteractive services, features, and applications. Such content andadditional information are downloaded over a network for display, use,and/or storage on client-side set-top boxes or similar devices. Becauseof bandwidth constraints, downloads that include audio and/or videoinformation, for example, are transmitted in a coded or compressedformat. In other words, to reduce the amount of data that istransmitted, the information is typically compressed from a first sizeto a second smaller size.

After the compressed/coded data is transmitted, the data isdecompressed/decoded into a semblance of the original audio and videoinformation. Unfortunately, because the compression-decompression cycleis typically lossy, the decompressed version of the audio and videoinformation differs from the original version of the audio and videoinformation. With video information, for example, such differences canresult in the presentation quality of the video information beingvisibly decreased.

Accordingly, for television-based entertainment systems, there is a needfor schemes and techniques to enable post-processing improvements tovideo information that has been adulterated in acompression-decompression cycle.

SUMMARY

Video deblocking can be implemented in video content processing anddelivery environments when displaying decompressed/decoded videoinformation. Discontinuities are identified and smoothed to reduceblockiness, for example at macroblock boundaries in conjunction withMoving Picture Expert Group (MPEG)-based schemes. In exemplaryimplementation(s), a client device detects discontinuities at or nearblock boundaries, determines which discontinuities are artificial, andsmooths at least those discontinuities that are determined to beartificial. These actions may be accomplished using (i) one or moreLaplacian of Gaussian (LoG) operations as applied to both an image and aquantization matrix and (ii) windowed comparisons between values and athreshold resulting from the LoG operations. Alternatively, theseactions may be accomplished using a spatio-temporally varying filter.Additionally, in an MPEG context, intra image units may be deblockedusing the former approach while non-intra image units may be deblockedusing the latter approach.

BRIEF DESCRIPTION OF THE DRAWINGS

The same numbers are used throughout the drawings to reference likeand/or corresponding aspects, features, and components.

FIG. 1 illustrates an exemplary television system architecture in whichthe systems and methods for video deblocking can be implemented.

FIG. 2 illustrates an exemplary client device, a television, and variousinput devices that interact with the client device.

FIG. 3 is a block diagram that illustrates components of the exemplaryclient devices shown in FIGS. 1 and 2.

FIG. 4 is a set of graphs that illustrate an exemplary discontinuity andsmoothing thereof.

FIG. 5 is a flow diagram that illustrates an exemplary method for videodeblocking.

FIG. 6 is a flow diagram that illustrates an exemplary process fordirectly deblocking video.

FIG. 7 illustrates an exemplary window for deblocking analysis around amacroblock boundary.

FIG. 8 illustrates an exemplary Laplacian of Gaussian (LoG) operation indiagrammatic form.

FIG. 9 is a graph that illustrates exemplary values resulting from anLoG operation on a quantization matrix.

FIG. 10 illustrates pixels around a macroblock boundary for an exemplaryvisual adjustment procedure.

FIGS. 11A, 11B, and 11C are exemplary diagrams of a portion of a videoinformation presentation across multiple frames of a group of pictures(GOP).

FIG. 12 is a graph that illustrates filters across macroblocks andmacroblock boundaries in an exemplary approach for indirectly deblockingvideo using spatial filtering.

FIGS. 13A and 13B illustrate an exemplary approach for indirectlydeblocking video using temporal filtering across multiple frames of aGOP.

FIG. 14 is a flow diagram of an exemplary approach toblock-type-dependent filtering under a Moving Pictures Expert Group(MPEG)-compliant compression/coding scheme.

DETAILED DESCRIPTION

The following discussion is directed to video content processing anddelivery systems, such as interactive TV networks, cable/satellitenetworks that utilize electronic program guides and other applications,and Web-enabled TV networks. Client devices in such systems range fromfull-resource clients with substantial memory and processing resources,such as TV-enabled personal computers and TV recorders equipped withhard-disks, to low-resource clients with limited memory and/orprocessing resources, such as traditional set-top boxes and personaldigital assistants (PDAs) or mobile phones. However, video deblocking asdescribed herein may additionally be used in other environments such asin streaming (e.g., over the Internet), generally in post-processing ofcompression and decompression cycles, and so forth. While aspects of thedescribed systems and methods can be used in any of these environmentsand for any types of client devices, they are described primarily in thecontext of the following exemplary environment.

Exemplary System Architecture

FIG. 1 illustrates an exemplary television entertainment system 100 thatis an architecture in which video deblocking may be implemented. System100 facilitates distribution of content and other information tomultiple viewers. System 100 includes one or more content providers 102,one or more other information providers 104, a content distributionsystem 106, and multiple client devices 108(1), 108(2), . . . , 108(N)coupled to content distribution system 106 via a broadcast network 110.

Content provider 102 includes a content server 112 and stored content114, such as movies, television programs, commercials, music, andsimilar audio and/or video content. Content server 112 controlsdistribution of stored content 114 from content provider 102 to contentdistribution system 106. Additionally, content server 112 may controldistribution of live content (e.g., content that was not previouslystored, such as live feeds) and/or content stored at other locations tocontent distribution system 106.

Other information provider 104 includes other information database 116and other information server 118. Other information database 116 storesinformation that may be provided to client devices 108. Such informationincludes software modules, files, images, text, executable programs,moving video, gaming or other interactive information, and so forth. Theinformation may also include content, especially content of anirregular, one-of-a-kind, or similar nature, or content from smallerindependent providers. Part or all of the information from otherinformation database 116 may ultimately be displayed to subscribersafter traversing one or more low-bandwidth transmission mediums,regardless of whether such transmission medium(s) are located upstreamor downstream of content distribution system 106. Other informationserver 118 processes the other information from other informationdatabase 116 prior to distribution to generate one or more files thatare optimized for, or at least capable of, transmission to contentdistribution system 106.

Content distribution system 106 includes a broadcast transmitter 128,one or more content processors 130, and one or more other informationprocessors 132. Broadcast transmitter 128 can alternatively be atransceiver if bidirectional communication is desired (e.g., insituations in which broadcast network 110 is a bidirectional network110). Broadcast transmitter 128 transmits (e.g., broadcasts) signals,such as cable/satellite television signals, across broadcast network110. Broadcast network 110 can include a cable television network, RF,microwave, satellite, and/or data network, such as the Internet, and mayalso include wired or wireless media using any transmission format orprotocol. Additionally, broadcast network 110 can be any type of network(including a bidirectional network), using any type of network topologyand any network communication protocol, and can be represented orotherwise implemented as a combination of two or more networks.

Content processor 130 processes the content received from contentprovider 102 prior to transmitting the content across broadcast network110. Similarly, other information processor 132 processes the otherinformation that is received from other information provider 104 priorto transmission of the other information across broadcast network 110. Aparticular content processor 130 may encode, or otherwise process, thereceived content into a format that is understood by the multiple clientdevices 108(1), 108(2), . . . , 108(N) that are coupled to broadcastnetwork 110. Although FIG. 1 shows a single content provider 102, asingle other information provider 104, and a single content distributionsystem 106, the exemplary system 100 can include any number of contentproviders and/or other information providers coupled to any number ofcontent distribution systems. Thus, content distribution system 106,content provider 102, and/or other information provider 104 areindividually or jointly representative of a headend service thatprovides content and other information to multiple subscribers.

Client devices 108 can be implemented in a number of ways. For example,a client device 108(1) receives content and other information from asatellite-based transmitter via a satellite dish 134. Client device108(1) is also referred to as a set-top box or a satellite receivingdevice. Client device 108(1) is coupled to a television 136(1) forpresenting the content and other information (e.g., audio information,video information, and/or data information) that are received by theclient device 108(1), as well as for presenting a graphical userinterface. A particular client device 108 can be coupled to any numberof televisions 136 and/or similar devices that can be implemented todisplay or otherwise render content. Similarly, any number of clientdevices 108 can be coupled to a single television 136.

Client device 108(2) is also coupled to receive content and otherinformation from broadcast network 110 and to provide the receivedcontent and other information to associated television 136(2). Clientdevice 168(N) is an example of a combination television 138 andintegrated set-top box 140. In this example, the various components andfunctionality of the set-top box are incorporated into the television,rather than using two separate devices. Set-top box 140 that isintegrated into television 138 can receive signals (e.g., broadcastsignals) via a satellite dish (similar to satellite dish 134) and/ordirectly via broadcast network 110. In alternate implementations, clientdevices 108 may receive signals via the Internet or any other network,especially those network mediums that are broadcast-capable. As isfurther described below, client devices 108 may also engage in videodeblocking prior to displaying video information (whether content videoinformation or other video information) that is received from a lowbandwidth medium, such as a memory storage, other client devices,possibly broadcast network 110 itself, and so forth.

The exemplary system 100 also includes information from othernetworks/network providers 142, which may provide information such asinformation streamed over the Internet, information received directlyfrom a provider of the information, and so forth. Information from othernetworks/network providers 142 may be accessible over broadcast network110 (i.e., a network that also provides content information and otherinformation from content distribution system 106). Alternatively,information from other networks/network providers 142 may be accessibleover a different network, including a wide area network (WAN), theInternet, a public or private telecommunications network, and so forth.

Exemplary Client Device

FIG. 2 illustrates an exemplary implementation 200 of a client device108 shown as a standalone unit that connects to a television 136 andcommunicates with various input devices 204, 206, and 208. Client device108 can be implemented in any number of embodiments, including as aset-top box, a satellite receiver, a TV recorder with a hard disk, adigital video record (DVR) and playback system, a game console, aninformation appliance, and so forth.

Client device 108 includes a wireless port 202, such as an infrared (IR)or Bluetooth wireless port, for receiving wireless communications from aremote control device 204, a handheld input device 206, or any otherwireless device, such as a wireless keyboard. Handheld input device 206can be a personal digital assistant (PDA), handheld computer, wirelessphone, or the like. Additionally, a wired keyboard 208 can be coupled tocommunicate with client device 108. In alternate embodiments, remotecontrol device 204, handheld device 206, and/or keyboard 208 may use anRF communication link or other mode of transmission to communicate withclient device 108.

Client device 108 receives one or more (e.g., broadcast) signals 210from one or more broadcast sources, such as from a satellite or a cableor a broadcast network, including a broadcast implementation of network110 (of FIG. 1). Client device 108 includes hardware and/or software forreceiving and decoding a broadcast signal 210, such as an NTSC, PAL,SECAM or other TV system video signal. Client device 108 also includeshardware and/or software for providing the user with a graphical userinterface by which the user can, for example, access various networkservices, configure client device 108, and perform other functions,including requesting video delivery.

Client device 108 can communicate with other devices via one or moreconnections including a conventional telephone line 212, an ISDN link214, a cable link 216, an Ethernet link 218, a DSL link 220, and thelike. Client device 108 may use any one or more of the variouscommunication links 212-220 at a particular instant to communicate withany number of other devices. For example, in addition to receivinginformation over any one of communication links 212-220, client device108 may provide (e.g., transmit) information over communication links212-220 or any other type of communication link. Such othercommunication links may include links capable of interfacing with alocal network such as a local area network (LAN), a Bluetooth® network,an IEEE 802.11b-compliant network, or other wired or wireless networktype. In such a local network environment, a client device 108 thatreceives information from broadcast network 110 may forward theinformation over the local network to one or more other client devices108. The forwarding may be effectuated, for example, over alower-bandwidth transmission medium that introduces blockiness.

Client device 108 generates video signal(s) 222 and audio signal(s) 224,both of which are communicated to television 136. Video signals 222 andaudio signals 224 can be communicated from client device 108 totelevision 136 via an RF (radio frequency) link, S-video link, compositevideo link, component video link, co-axial cable link, or othercommunication link. The video signals 222 may include deblocked videosignals, for example. Although not shown in FIG. 2, client device 108may include one or more lights or other indicators identifying thecurrent status of the device. Additionally, the client device mayinclude one or more control buttons, switches, or other selectablecontrols for controlling operation of the device.

FIG. 3 illustrates selected components of exemplary client device 108shown in FIGS. 1 and 2. Client device 108 includes a first tuner 300 andan optional second tuner 302. The tuners 300 and 302 are representativeof one or more in-band tuners that tune to various frequencies orchannels to receive television signals, as well as at least oneout-of-bound (OOB) tuner that tunes to the broadcast channel(s) overwhich data information is broadcast (e.g., carouseled or otherwisetransmitted) to client device 108.

Client device 108 also includes one or more processors 304 which processvarious instructions to control the operation of client device 108 andto communicate with other electronic and computing devices. Clientdevice 108 can be implemented with one or more memory components,examples of which include a random access memory (RAM) 306, a disk drive308, another mass storage component 310, and a non-volatile memory 312(e.g., ROM, Flash, EPROM, EEPROM, etc.). The memory components (e.g.,RAM 306, disk drive 308, mass storage 310, and non-volatile memory 312)store various instructions and/or information such as received content,programs, configuration information for client device 108, graphicaluser interface information, and/or video data or information.

Alternative implementations of client device 108 can include a range ofprocessing and memory capabilities, and may include more or fewer typesof memory components than those illustrated in FIG. 3. For example,full-resource clients can be implemented with substantial memory andprocessing resources, including the disk drive 308 to store content forreplay by the viewer. Low-resource clients, however, may have limitedprocessing and memory capabilities, such as a limited amount of RAM 306,no disk drive 308, and limited processing capabilities of a processor304.

An operating system 314 and one or more programs as represented bygeneral instructions 316 may be stored in non-volatile memory 312(and/or other memory component(s)) and executed on processor 304 toprovide a runtime environment. A runtime environment facilitatesextensibility of client device 108 by allowing various interfaces to bedefined that, in turn, allow the programs to interact with client device108. Although these programs may be installed when client device 108 ismanufactured, they may also be received via broadcast network 110 fromcontent distribution system 106 (of FIG. 1). Also stored in non-volatilememory 312 (and/or other memory component(s)) areelectronically-executable instructions for video deblocking 318, asdescribed further herein.

Client device 108 also includes a decoder 320 to decode a broadcastvideo signal, such as an NTSC, PAL, SECAM or other TV system videosignal. Processor 304, along with tuner(s) 300 and 302 and/or decoder320, also enables client device 108 to reconstruct audio and video froman MPEG-2 stream or other digital packet signal, whether compressed oruncompressed. Client device 108 can also include other componentspertaining to a television entertainment system which are notillustrated in this example. For instance, client device 108 can includea user interface application and user interface lights, buttons,controls, and the like to facilitate viewer interaction with the device.

Client device 108 further includes a wireless interface 322, a networkinterface 324, a serial and/or parallel interface 326, and a modem 328.Wireless interface 322 allows client device 108 to receive inputcommands and other information from a user-operated input device, suchas from a remote control device or from another IR, Bluetooth, orsimilar RF input device. Network interface 324 and serial and/orparallel interface 326 allows client device 108 to interact andcommunicate with other electronic and computing devices via variouscommunication links, including local network communication links toother client devices 108. Although not shown, client device 108 may alsoinclude other types of data communication interfaces to communicate withother devices. Modem 328 facilitates communication by client device 108with other electronic and computing devices via a conventional telephoneline.

Client device 108 also includes an audio output 330 and a video output332 that provide signals to a television or other device that processesand/or displays or otherwise renders the audio and video information,including deblocked video information. Although shown separately, someof the components of client device 108 may be implemented together in anapplication specific integrated circuit (ASIC). For example,electronically-executable instructions (318) and associated processingabilities for video deblocking may be integrated together onto one ormore ASICs instead of using more-general memory (e.g., 306, 312, etc.)and/or processing (e.g., 304, 320, etc.) resources of a client device(108).

Additionally, a system bus (not shown) typically connects the variouscomponents within client device 108. A system bus can be implemented asone or more of any of several types of bus structures, including amemory bus or memory controller, a peripheral bus, an acceleratedgraphics port, or a local bus using any of a variety of busarchitectures. By way of example, such architectures can include anIndustry Standard Architecture (ISA) bus, a Micro Channel Architecture(MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics StandardsAssociation (VESA) local bus, and a Peripheral Component Interconnects(PCI) bus also known as a Mezzanine bus.

Video Deblocking

Video information is often compressed or otherwise coded to facilitatethe handling thereof. The coding of the video information into videodata can reduce the bandwidth required to display, store, transmit, orotherwise manipulate the video information. For example, it can bebeneficial to reduce the bit rate of video information for storing avideo stream to disk or streaming it over a noisy, bandwidth-limitedchannel. However, compression and decompression of the video informationtypically causes visual artifacts that degrade the visual quality. Oneartifact that degrades visual quality is blockiness. In other words, lowbit rate video information is often plagued with blockiness thatseriously degrades visual quality. Video deblocking, on the other hand,usually entails reducing such blocky artifacts to enhance the visualpresentation quality of the video information. Implementing a gooddeblocking mechanism helps to mitigate the deleterious byproducts of alower bit rate video stream and thereby justifies using a lower bit ratethat enables the aforementioned low-bandwidth applications.

Deblocking is implemented after previously encoded video data is decodedinto at least a semblance of the original video information. One set ofcoding standards that may be used on the original video information areMoving Picture Expert Group (MPEG)-compliant standards. MPEG-basedencoding/decoding algorithms involve performing a discrete cosinetransform (DCT) on original video information and quantizing the DCTcoefficients thereof. The resulting video data is de-quantized and aninverse DCT is performed to reacquire at least a semblance of theoriginal video information. Reducing the bit rate of the video datastream may be achieved by more coarsely quantizing the DCT coefficients.In doing so, however, blockiness that is introduced along macroblockboundaries tends to seriously degrade the visual quality. The visualpresentation quality may be at least partially restored by usingdeblocking during post-processing. Specifically, the deblocking may beused to reduce the blockiness around macroblock boundaries.

FIG. 4 is a set of graphs 400 that illustrate an exemplary discontinuityand smoothing thereof. Graphs 400 include graphs 400A, 400B, 400C, and400D. Each of graphs 400A-400D includes two macroblocks of a videopicture. Although the macroblocks are shown as being rectangular, actualmacroblocks may instead be square. Graph 400A illustrates original videoinformation 402 (as a solid line). There is no break in original videoinformation 402 between the left macroblock and the right macroblock.

Graph 400B illustrates original video information 402 and adds theretoan illustration of discontinuity 404 (as a large dashed line).Discontinuity 404 represents an exemplary discontinuity or break in theseamlessness of original video information 402. This discontinuityarises at the macroblock boundary of the left and right macroblocks as aresult of the encoding/decoding process. Discontinuity 404 may bevisible as a blocky artifact. In fact, discontinuity 404 along withother discontinuities in the overall picture may be visible as ablockiness that reduces the overall visual quality.

Graph 400C illustrates original video information 402 and adds theretoan illustration of smoothed video information 406 (as a small dashedline). Smoothed video information 406 results from application of videodeblocking to discontinuity 404. While smoothed video information 406 isnot likely to be an exact replica of original video information 402,smoothed video information 406 reduces the blocky artifact ofdiscontinuity 404 and generally more-closely approximates original videoinformation 402. It should be noted that smoothed video information 406is representative of deblocking mechanisms/approaches in general,including visual adjustment, filter smoothing, spatial-temporalfiltering, and so forth.

Graph 400D includes original video information 402, discontinuity 404,and smoothed video information 406. All three are included together toillustrate (i) the discontinuity that may be formed from coarsequantization of original video information or other deleteriouscoding/decoding factors and (ii) the resulting video information afterthe discontinuity has been smoothed in post-processing. Thispost-processing video deblocking is described further herein.

Specifically, the description herein focuses on deblocking in anMPEG-based environment in which discontinuities appear at macroblockboundaries. However, deblocking may be employed afterdecompression/decoding in accordance with other standards and approachesand along other boundaries. It should therefore be understood thatdeblocking may be employed with video data/information in general andregardless of the size/type of blocks that are employed to segregate thevideo picture.

Although the term “discontinuity” is used herein, other such termsinclude “edge” and “boundary”. An edge, boundary, or discontinuity in avideo picture may be “artificial” or “real”. Artificial discontinuitiesarise, for example, from coarse quantization of DCT coefficients. Otherterms for artificial discontinuities include, but are not limited to,quantization discontinuities, false discontinuities, fakediscontinuities, and quantization noise discontinuities. Realdiscontinuities, on the other hand, arise from true boundaries in avideo picture from one element to another element within the picture.Other terms for real discontinuities include, but are not limited to,actual discontinuities, true discontinuities, and naturaldiscontinuities. These real discontinuities may also lie along amacroblock boundary. Consequently, if smoothing is performed on alldiscontinuities at all macroblock boundaries, some real discontinuitieswill be smoothed. This causes the picture to be fuzzy and less distinct.However, if discontinuities at macroblock boundaries can be separatedinto real discontinuities and artificial discontinuities, then smoothingmay be performed on the artificial discontinuities.

Methods for Video Deblocking

Video deblocking may be described in the general context ofelectronically-executable instructions. Generally,electronically-executable instructions include routines, programs,objects, components, data structures, and the like that performparticular functions or implement particular abstract data types. Videodeblocking may also be practiced in distributed computing environmentswhere functions are performed by remote processing devices that arelinked through a communications network. In a distributed computingenvironment, electronically-executable instructions may be located inboth local and remote storage media.

The methods and processes of FIGS. 5, 6, and 14 are illustrated in flowdiagrams divided into multiple blocks. However, the order in which themethods and processes are described is not intended to be construed as alimitation, and any number of the described blocks can be combined inany order to implement one or more methods or processes for videodeblocking. Furthermore, although the methods and processes aredescribed below with reference to the television entertainmentenvironments 100 and 200 and client devices 108 where applicable, themethods and processes can be implemented in any suitable hardware,software, firmware, or combination thereof and using any suitablemathematical alternatives.

FIG. 5 is a flow diagram 500 that illustrates an exemplary method forvideo deblocking. Flow diagram 500 includes blocks 502, 504, and 506that may be implemented, for instance, by client devices 108 (of FIGS.1-3). Blocks 502, 504, and 506 may correspond toelectronically-executable instructions that are stored in one or morememories of a client device 108 and executed using one or moreprocessors thereof. At block 502, a discontinuity is detected. Forexample, macroblock boundaries are analyzed to detect whether adiscontinuity exists for video information between a first macroblockboundary and a second macroblock boundary. An exemplary window around amacroblock boundary for such an analysis is described further below withreference to FIG. 7.

At block 504, the discontinuity is examined to determine whether it is areal discontinuity or an artificial discontinuity. Real edges of actualpicture elements may fall on a macroblock boundary, but smoothing realedges usually decreases video presentation quality. Exemplary mechanismsfor differentiating between real discontinuities and artificialdiscontinuities are described further below especially with reference toFIGS. 8 and 9.

At block 506, the artificial discontinuity or discontinuities aresmoothed. The smoothing of the artificial discontinuities reducesvisually-apparent blockiness. It should be understood that smoothing mayencompass a variety of mechanisms/approaches designed to reduce oreliminate the visibility of discontinuities. Exemplary mechanisms forsmoothing artificial discontinuities are described further belowespecially with reference to FIG. 10 et seq.

Flow diagram 500 is directed to both direct deblocking and indirectdeblocking. Direct deblocking of video is described especially withreference to FIGS. 6-10. Indirect deblocking of video is describedespecially with reference to FIGS. 11A-13B. The description of FIG. 14below is directed to a video deblocking approach that utilizes bothdirect and indirect deblocking mechanisms.

FIG. 6 is a flow diagram 600 that illustrates an exemplary process fordirectly deblocking video. Flow diagram 600 includes six (6) blocks602-612. Blocks 602 and 604 pertain to input data for the process offlow diagram 600. Block 602 provides the scaled quantization matrix thatwas used to quantize the original video information during thecoding/compressing process. This scaled quantization matrix of block 602may be related to, for example, the “quant_matrix” parameter ofMPEG-compliant coding schemes. The quant_matrix is determined by theencoder and encoded into the bit stream. A scaled quant matrix may becreated as the product of the quant matrix and the quantization scale,which is another parameter that is encoded into the bit stream by theencoder. Block 604 provides the image that is formed from the decodedvideo information. As such, the image may include artificialdiscontinuities (e.g., at macroblock boundaries) and realdiscontinuities (e.g., either at or away from macroblock boundaries).The artificial discontinuities at the macroblock boundaries can arisefrom noise attributable to the quantization matrix during thequantization process.

The scaled quantization matrix of block 602 and the image videoinformation of block 604 are submitted to block 606. Block 606 performsa Laplacian of Gaussian (LoG) operation on each of the scaledquantization matrix of block 602 and the image video information ofblock 604 to produce two different results. The LoG operation may bedefined as convolving the argument with the LoG kernel. From the twodifferent results of applying the LoG operation, the type ofdiscontinuity is determined at block 608. For example, the type ofdiscontinuity (if any) at the macroblock boundary under consideration isdetermined by comparing the result of the LoG operation as performed onthe image of block 604 with a (quantization) threshold determinedresponsive to the result of the LoG operation as performed on the scaledquantization matrix of block 602. The LoG operation of block 606 and thediscontinuity type determination of block 608 are described furtherbelow especially with reference to FIGS. 8 and 9.

If the discontinuity at the macroblock boundary is determined to be areal discontinuity (at block 608), block 610 may optionally perform avisual adjustment procedure at the macroblock boundary. An exemplaryvisual adjustment procedure is described further below especially withreference to FIG. 10. If, on the other hand, the discontinuity at themacroblock boundary is determined to be an artificial discontinuity (atblock 608), block 612 performs a filter smoothing procedure at themacroblock boundary. An exemplary filter smoothing procedure isdescribed further below primarily after the description of FIG. 10.

FIG. 7 illustrates an exemplary window 702 for deblocking analysisaround a macroblock boundary 706. Two macroblocks 708L and 708R areillustrated. Each macroblock 708 is surrounded by four (4) macroblockboundaries 706. Within each macroblock 708 are a set of pixels. Althoughmacroblocks 708 are shown as being blocks of eight-by-eight (8×8)pixels, they may alternatively be blocks of 16×16 pixels, 24×24 pixels,30×30 pixels, or any other size. Because the illustrated macroblocks areof an eight-by-eight (8×8) size, window 702 includes sixteen (16) pixelsto be analyzed for possible discontinuities at the macroblock boundary706 that window 702 surrounds. However, the width of window 702 mayalternatively be greater than two pixels (i.e., the width can be morethan one pixel within each macroblock 708).

Window 702 is located between the left macroblock 708L and the rightmacroblock 708R. Similar “vertical” windows 702 are used at the other“vertical” macroblock boundaries 706. Furthermore, “horizontal” windows(not shown) are used at macroblock boundaries between upper macroblocksand adjacent lower macroblocks. In other words, to address atwo-dimensional (2-D) image, video deblocking is performed in both thevertical direction and the horizontal direction.

To detect a discontinuity within a window 702, the following algorithmmay be employed: Let Q represent the quantization matrix used inquantizing an image I. Because quantization is usually performed in theDCT domain, the effect of using Q in the time domain is firstdetermined. Thus, an inverse DCT is performed on Q to generate Q_(t).Next, the discontinuities (e.g., edges, boundaries, etc.) are detectedin the image L Any such discontinuities may be detected using theLaplacian of Gaussian (LoG) operator to filter I and generate I_(LG).The LoG operator may be considered a concatenation of the Laplacianoperator, which is given by:${L( {x,y} )} = {{\nabla^{2}{f( {x,y} )}} = {\frac{\partial^{2}{f( {x,y} )}}{\partial x^{2}} + \frac{\partial^{2}{f( {x,y} )}}{\partial y^{2}}}}$and the Gaussian operator, which is given by:${G( {x,y} )} = {\frac{1}{2\pi}{\mathbb{e}}^{{- \frac{1}{2\sigma^{2}}}{({x^{2} + y^{2}})}}}$thereby yielding:${{LoG}( {x,y} )} = {{- {\frac{1}{{\pi\sigma}^{4}}\lbrack {1 - \frac{x^{2} + y^{2}}{2\sigma^{2}}} \rbrack}}{{\mathbb{e}}^{- \frac{x^{2} + y^{2}}{2\sigma^{2}}}.}}$The filtered image is therefore given by:I _(LG) =LoG*I.

FIG. 8 illustrates an exemplary Laplacian of Gaussian (LoG) operation800 in diagrammatic form. An image 802 is filtered using LoG operation804 to produce an edge-biased image 806. After applying the LoGoperation to image I, discontinuities (e.g., edges) can be detected ineach window 702 surrounding every analyzed macroblock boundary 706.However, it is still to be determined whether each discontinuity in eachwindow 702 is a real discontinuity (e.g., from an edge of an imageelement) or an artificial discontinuity (e.g., from quantization orother deleterious compression/decompression factors). The LoG operationdetects both kinds of discontinuities, but it does not clearlydifferentiate between them by itself

A variable that may be termed “quant_threshold” is created to determinewhich discontinuities are artificial. The amount of blockinessattributable to quantization noise is effectively assessed. In order toassess the amount of blockiness attributable to quantization, thetime-domain quantization matrix divided by two ( Q_(t)/2 is passedthrough the LoG filter to obtain Q_(tLG). The quantization matrix Q_(t)is divided by 2 because half the quantization matrix Q_(t) representsthe actual loss of data due to round-off/truncation. After obtainingQ_(tLG) from passing ( Q_(t)/2 through the LoG filter, the maximum andminimum values of Q_(tLG) are examined to determine the quant_threshold.

FIG. 9 is a graph 900 that illustrates exemplary values Q_(tLG)resulting from a LoG operation on a time-domain quantization matrixQ_(t). Thus, a plot 902 graphs different quantization matrix values(along the abscissa axis) versus corresponding LoG operation outputvalues Q_(tLG) (along the ordinate axis). Maximum absolute values forthe Q_(tLG) values are denoted as one hundred percent (100%). Anyportion of the 100% value may be selected as the threshold between realdiscontinuities and artificial discontinuities. The threshold may beselected, for example, such that the majority of the visuallydispleasing artificial discontinuities are subject to smoothing whilefew if any of the real discontinuities are subject to smoothing (thereinmitigating “fuzzying” of the original image). An optimum value for thethreshold for any given type of video information may be selectedthrough experimental viewing of deblocked video. In the illustratedgraph 900, eighty-five percent (85%) is selected as the thresholdbetween real discontinuities and artificial discontinuities.Furthermore, this threshold may be tunable subject to user/viewerpreferences.

After the threshold between real discontinuities and artificialdiscontinuities is selected, the threshold is used in conjunction withan analysis of macroblock boundaries 706 in windows 702. The analysis isfocused in windows 702 because blockiness that is due to quantizationnoise is centered on macroblock boundaries 706. The values of I_(LG) arecompared to the threshold within each window 702. If more than a certainfraction, η, of the I_(LG) values in a window 702 are above the selectedthreshold, then that discontinuity is labeled as a real discontinuity.If not, then that discontinuity is labeled as an artificialdiscontinuity. The fraction, η, may also be selected experimentally, forexample by viewing different types of video that has been deblockedusing different values for the fraction, η, until a visually-appealingvalue is selected. The parameter, η, may be experimentally determinedand fine tuned to maximize visual quality.

After discontinuities have been (i) detected and (ii) determined to be(and “labeled” as) either real discontinuities or artificialdiscontinuities, appropriate deblocking procedures can be implemented.As described above with reference to blocks 608, 610, and 612 (of FIG.6), real discontinuities may be subjected to a visual adjustmentprocedure, and artificial discontinuities may be subjected to a filtersmoothing procedure. In an exemplary visual adjustment procedure, pixelvalues are adjusted slightly by moving them towards each other by anamount proportional to the threshold so as to reduce the effects ofquantization.

FIG. 10 illustrates pixels 704 around a macroblock boundary 706 for anexemplary visual adjustment procedure. Eight (8) adjacent pixels 704from p0 to p7 are divided by the macroblock boundary 706 that is betweenleft macroblock 708L and right macroblock 708R. Pixels p0, p1, p2, andp3 are in left macroblock 708L. Pixels p4, p5, p6, and p7 are in rightmacroblock 708R. The values of the pixels p0 to p7 are adjusted slightlyby moving them towards each other by an amount proportional to theselected threshold so as to reduce the effects of quantization. Thisadjustment is performed recursively to the adjacent pixels.

The exemplary visual adjustment procedure for real discontinuities, whenperformed, may be effectuated as follows: Let p=[p₀p₁ . . . p₇]represent the pixels 704 around a macroblock boundary 706 as illustratedin FIG. 10. The following series of operations are performed:$d = {\min( {\frac{p_{3} - p_{4}}{2},{quant\_ threshold}} )}$p₃^(′) = p₃ − d p₄^(′) = p₄ + d$p_{2}^{\prime} = \frac{p_{2} + p_{3}^{\prime}}{2}$$p_{5}^{\prime} = {\frac{p_{5} + p_{4}^{\prime}}{2}\quad\ldots}$

-   -   where decoded, time-domain pixels are designated by “p” (p        without the prime) and the adjusted pixels are represented by        “p′” (p with the prime).

If, on the other hand, a detected discontinuity has been determined tobe, and “labeled” as, an artificial discontinuity, then a filtersmoothing procedure is performed. In an exemplary filter smoothingprocedure, pixels 704 (of FIG. 7) of a window 702 are filtered aroundboundary 706 with a smoothing filter. In other words, a pixel array, p,is filtered with a smoothing filter such as stretched and/or scaledversion(s) of the Gaussian or tent filter. An example of a Gaussian/tentfilter is:${G\lbrack n\rbrack} = {\frac{1}{2\pi}{\mathbb{e}}^{{- \frac{1}{2\sigma^{2}}}n^{2}}}$${T\lbrack n\rbrack} = \{ \begin{matrix}{1 - {\frac{n}{5}}} & {{- 5} \leq n \leq 5} \\0 & {otherwise}\end{matrix} $In general, any smoothing filter can be used with the degree ofsmoothing set to accommodate the desired amount of blurring.

In the second approach to video deblocking, a less rigid and moreindirect mechanism is used to detect discontinuities, to differentiatebetween real and artificial discontinuities, and to smooth the detecteddiscontinuities. This second mechanism includes both spatial andtemporal aspects. In a spatial aspect, the mechanism entails associatingan increased likelihood of needing deblocking (e.g., of a discontinuitybeing an artificial discontinuity) with increased proximity to amacroblock boundary. In a temporal aspect, the mechanism entailsassociating the likelihood of blockiness at a macroblock boundary to theposition of the picture within a GOP. For example, it is assumed that anI-frame of the GOP is more susceptible to blockiness at the macroblockboundary. On the other hand, a B-frame downstream in the GOP is lesslikely to experience blockiness at the macroblock boundary.

FIGS. 11A, 11B, and 11C are exemplary diagrams 1100 of a portion ofvideo information that is presented across multiple frames of a GOP. Inan MPEG-compliant coding scheme, frames are organized into GOPs thatstart with an intra (I) frame and are followed by predicted (P) framesand bi-directional (B) frames/modes. I-frames are usually moresusceptible to blockiness at macroblock boundaries than those framesthat are downstream in the GOP as is explained by the description of thediagrams 1100A-C. Specifically, diagram 1100A represents an intra-framewith many macroblocks 708 and macroblock boundaries 706.

Diagrams 1100B and 1100C represent non-intra frames. Each non-intrablock in non-intra frames includes two components: the predicted termand the difference term. Typically, the predicted term dominates, andthe difference term is relatively small. Because the predicted term isusually not macroblock aligned in the reference frame (as shown indiagram 110B), any macroblocking effects in the reference frame are seenin the interior of the reconstructed macroblock in the current frame.Moreover, as this effect occurs over several generations of prediction,the magnitude and location of blockiness tends to be diffused (as shownin diagram 1100C). In other words, downstream in a GOP, images tend tobe both less crisp as well as less blocky. Thus, the position of a framein a GOP, or GOP position number (GPN), tends to serve as an indicatorof the extent of blockiness. This tendency is reflected in the temporalaspect of the second mechanism. Similarly, for the spatial aspect of thesecond mechanism, the extent of quantization-caused blockiness tends toincrease as pixel location approaches a macroblock boundary.

The second mechanism thus involves filtering the image with aspatio-temporally varying filter. The degree of filtering increases forpixels near macroblock boundaries and decreases for pixels away frommacroblock boundaries. This spatial filtering is described further belowwith reference to FIG. 12. Also, the degree of filtering is higher atthe beginning of a GOP while the degree of filtering decreasesdownstream in the GOP. This temporal filtering is described furtherbelow with reference to FIGS. 13A and 13B. Combining the spatialfiltering with the temporal filtering creates a spatio-temporallyvarying filter that indirectly detects discontinuities, differentiatesbetween discontinuity types, and smooths the detected discontinuities.

FIG. 12 is a graph 1200 that illustrates filters 1202 across macroblocks708 and macroblock boundaries 706 in an exemplary approach forindirectly deblocking video using spatial filtering. Filters 1202include three (3) different filtering zones labeled zone #1, zone #2,and zone #3. While each of the three different filtering zones may havefilters of similar or identical energy levels (to avoid changing imageintensity), each of the filters of the three different filtering zonesfilters pixels of the macroblocks 708 differently. Filtering zone #1filters to the greatest degree in an area around macroblock boundaries706. Filtering zone #3 filters the least in an area around the center ofmacroblocks 708. Filters of filtering zones #3 may be implemented as animpulse function so that the pixels are changed very slightly or not atall. The centers of macroblocks 708 are the least likely to exhibitblockiness due to quantization noise. Located spatially between thefiltering zones #1 and #3 are filtering zones #2. The filters offiltering zones #2 filter the corresponding physically-intermediatepixels to a degree that is also intermediate to that of the filters offiltering zones #1 and #3.

Using multiple filtering zones, the pixels may be averaged, for example,to a greater extent near macroblock boundaries and to gradually lesserextents for pixels that are gradually nearer the middle of macroblocks.Although filter 1202 includes an exemplary three filtering zones, fouror more or less than three filtering zones may alternatively beemployed. An example of a suitable spatial filter is:${G_{\sigma}\lbrack n\rbrack} = {\frac{1}{2\pi}{\mathbb{e}}^{{- \frac{1}{2\sigma^{2}}}n^{2}}}$

-   -   where we can distinguish the various zones on the basis of the        chosen value for σ, for instance, as:    -   Zone 1: σ=1;    -   Zone 2: σ=0.707; and    -   Zone 3: σ=0.5.        Furthermore, the filters can be normalized in order to achieve a        gain of 1 (or any desired value). Such spatial filtering thus        concentrates filtering near macroblock boundaries and away from        the center of macroblocks. Using a spatial filter designed        accordingly effectively tends to indirectly filter blockiness        due to quantization noise.

FIGS. 13A and 13B illustrate an exemplary approach for indirectlydeblocking video using temporal filtering across multiple frames of aGOP. The extent of spatial filtering that is necessary and/or desirablearound macroblock boundaries tends to decrease as frame position numberproceeds through a GOP, as is explained above with reference to FIGS.11A-11C. Chart 1300A illustrates a GOP and exemplary temporal filteringcoefficients designated as α. The GOP position number (GPN) runs from 1to n, where n may be any natural number, but is often set to fifteen(15). The MPEG frames are designated by I, B, or P in an exemplarysequence that runs: I, B, B, B, P, B, B, B, P, B, B, B, P, B, B. Othersequences may alternatively be used. Three different sets of temporalfiltering coefficients α are included and are designated as α1, α2, andα3.

The temporal filtering coefficient α may be applied to the spatialfiltering as described above with reference to FIG. 12. The temporalfiltering coefficient α serves to reduce the filtering by, and thereforethe impact of, the spatial filter 1202 as frames progress downwardthrough the GOP. Consequently, each α may be determined responsive tothe GPN of the frame under consideration. Table 1300B provides a generalGPN-dependent formula for determining an α for each of the threetemporal filtering coefficient α examples. Each of the a examplesdecrease in value as the GPN increases to reduce the impact of thespatial filter. The rapidity at which each α decreases in value as theGPN increases from 1 to n scales from a low with α1 to a high with α3.Besides the formulas provided in table 1300B, other formulas fordetermining α may alternatively be employed. Such other formulas, oreven other sets of α values that are not necessarily formulaic, need notuse an actual GPN numeral to set the α.

The temporally varying parameter, α, may be used to vary the degree offiltering using, for example, the following approach:${G_{\sigma}\lbrack {\alpha,n} \rbrack} = {{K \cdot \frac{1}{2\pi}}{\mathbb{e}}^{{- \frac{1}{2{({\alpha\sigma})}^{2}}}n^{2}}}$where the number K is chosen to normalize the energy in the filter.Also, if the filtering ability of the spatial filter 1202 is inverselyproportional to α, then the α formula may be set up so as to increasewith increasing GPN.

An application of the α formulas of table 1300B are provided under theappropriate GPNs of chart 1300A. For example, given an n=15, α1 is theset of numbers {1, 14/15, 13/15, 12/15, 11/15, . . . , 2/15, 1/15}; α2is the set of numbers {1, 1/2, 1/3, 1/4, 1/5, . . . , 1/14, 1/15}; andα3 is the set of numbers {1, 1/4, 1/9, 1/16, 1/25, . . . , 1/196,1/225}. An α formula or set of values that results in visually-pleasingvideo may be determined experimentally for any given type of videoinformation. Use of the temporal filtering coefficient α as a modifierto a spatial filter 1202 enables application of a spatio-temporallyvarying filter that indirectly detects, differentiates between types of,and smoothes discontinuities.

FIG. 14 is a flow diagram 1400 of an exemplary approach toblock-type-dependent filtering under an MPEG-compliantcompression/coding scheme. The second mechanism, as described above withparticular reference to FIGS. 11A-13B, may be applied to all frames of aGOP. Alternatively, the approach of flow diagram 1400 may be employed inwhich the video deblocking mechanism that is being applied depends onthe frame/block type. Flow diagram 1400 is described in terms of blockssuch as macroblocks, but it may instead be implemented on a frame-onlybasis. More generally, blocks, frames, macroblocks, etc. may beconsidered units of an image.

As noted above especially in the descriptions of FIGS. 11A-11C and 13A,each MPEG GOP starts with an I-frame. This I-frame has relatively hardand certain macroblock boundaries. Non-I-frames (e.g., B-frames andP-frames), on the other hand, may have discontinuities that are notmacroblock aligned in the current frame. To handle this difference, thefirst mechanism (which is more direct) may be applied to I-frames whilethe second mechanism (which is more indirect) may be applied tonon-I-frames. Thus, the deblocking under such a scheme isframe-type-dependent. Under MPEG coding, however, individual macroblocksin a non-I-frame may be equivalent to an I-frame. In other words, someblocks in P-frames and B-frames are unpredicted, have no inherentdiscontinuity blurring, and are macroblock aligned in the current frame.These I-blocks in P-frames and B-frames can thus be treated using themore direct first mechanism.

Accordingly, a determination as to which mechanism is to be used underan MPEG-compliant coding scheme may be block-type-dependent. In flowdiagram 1400, a flow diagram block 1402 determines whether a block underconsideration is (i) an intra block or (ii) a predicted or abi-directional block. For example, prior to actual smoothing, eachmacroblock of a frame of video information may be analyzed to determinewhether it is an I-block or a P-block/B-block. If it is determined to bean I-block, then a visual adjustment procedure or a filter smoothingprocedure is implemented at flow diagram block 1404. Whether a visualadjustment procedure or a filter smoothing procedure is implemented isdependent on discontinuity type. As described further above withreference especially to FIGS. 6-10, the visual adjustment procedure isapplied to real discontinuities and the filter smoothing procedure isapplied to artificial discontinuities.

If, on the other hand, the block under consideration is determined to bea P-block or a B-block (e.g., a non-I-block), then a spatio-temporallyvarying filtering procedure is implemented for the block underconsideration at flow diagram block 1406. The extent of filtering forany given pixel or set of pixels is dependent on the spatial position ofthe pixel or pixels within the block. The extent of filtering for thegiven pixel or set of pixels is also dependent on the temporal-based GPNof the frame of which the block forms a part. Such a spatio-temporallyvarying filtering procedure is described further above with referenceespecially to FIGS. 11A-13B. In manners in accordance with flow diagram1400, the different types of MPEG frames or blocks may therefore behandled using different deblocking mechanisms.

CONCLUSION

Although systems and methods have been described in language specific tostructural features and/or methods, it is to be understood that theinvention defined in the appended claims is not necessarily limited tothe specific features or methods described. Rather, the specificfeatures and methods are disclosed as exemplary forms of implementingthe claimed invention.

1. A method for providing video deblocking, comprising: applying aLaplacian of Gaussian (LoG) operation on an image to produce a firstplurality of values, the image being previously encoded using aquantization matrix; applying the LoG operation on the quantizationmatrix to produce a second plurality of values; determining a thresholdresponsive to the second plurality of values; comparing at least aportion of the first plurality of values to the threshold; ascertaininga number of the at least a portion of the first plurality of values thatexceed the threshold; and if the number is less than a predeterminedfraction, then labeling a discontinuity that is associated with the atleast a portion of the first plurality of values as an artificialdiscontinuity:
 2. The method as recited in claim 1, further comprising:if the number is greater than the predetermined fraction, then labelingthe discontinuity that is associated with the at least a portion of thefirst plurality of values as a real discontinuity.
 3. The method asrecited in claim 2, further comprising: performing a visual adjustmentprocedure on the real discontinuity.
 4. The method as recited in claim1, wherein the at least a portion of the first plurality of valuescorresponds to pixels of the image that are proximate to at least onemacroblock boundary.
 5. The method as recited in claim 1, wherein thequantization matrix comprises a scaled quantization matrix.
 6. Themethod as recited in claim 1, wherein the action of determining athreshold responsive to the second plurality of values comprises:selecting a percentage of an absolute maximum value of the secondplurality of values as the threshold.
 7. The method as recited in claim1, further comprising: performing a filter smoothing procedure on theartificial discontinuity.
 8. One or more electronically-accessible mediacomprising electronically-executable instructions that, when executed,direct an electronic device to perform the method as recited in claim 1.9. An arrangement capable of deblocking video, comprising: detectingmeans for detecting one or more discontinuities at macroblock boundariesof a video image; determining means for determining whether the one ormore discontinuities include one or more artificial discontinuities andwhether the one or more discontinuities include one or more realdiscontinuities; and smoothing means for smoothing the one or morediscontinuities in dependence on whether each discontinuity of the oneor more discontinuities comprises an artificial discontinuity or a realdiscontinuity.
 10. The arrangement as recited in claim 9, wherein thearrangement comprises a client device of a television-basedentertainment environment.
 11. The arrangement as recited in claim 9,wherein the detecting means utilizes a Laplacian of Gaussian (LoG)operation on the video image and the determining means utilizes the LoGoperation on a quantization matrix.
 12. The arrangement as recited inclaim 11, wherein the determining means includes selecting means forselecting a discontinuity threshold from a plurality of results of theLoG operation on the quantization matrix.
 13. The arrangement as recitedin claim 11, wherein the quantization matrix comprises a scaledquantization matrix.
 14. The arrangement as recited in claim 9, whereinthe smoothing means includes visual adjustment means for visuallyadjusting each discontinuity that comprises a real discontinuity andfilter smoothing means for filter smoothing each discontinuity thatcomprises an artificial discontinuity with at least one smoothingfilter.
 15. The arrangement as recited in claim 14, wherein the at leastone smoothing filter comprises at least one of a tent filter and aGaussian filter.
 16. A client device for a television-basedentertainment environment, the client device configured to performactions comprising: detecting one or more discontinuities at macroblockboundaries of a video image; determining whether the one or morediscontinuities include one or more artificial discontinuities andwhether the one or more discontinuities include one or more realdiscontinuities; smoothing the one or more artificial discontinuitiesusing a filter smoothing procedure; and smoothing the one or more realdiscontinuities using a visual adjustment procedure.
 17. The clientdevice as recited in claim 16, wherein the client device is furtherconfigured to detect the one or more discontinuities by applying aLaplacian of Gaussian (LoG) operation on the video image.
 18. The clientdevice as recited in claim 16, wherein the client device is furtherconfigured to determine whether the one or more discontinuities includeone or more artificial discontinuities and whether the one or morediscontinuities include one or more real discontinuities using aLaplacian of Gaussian (LoG) operation on a quantization matrix.